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Abstract 

Let Qi, i = 1, be real nondegenerate indefinite quadratic forms in 

d variables. We investigate under what conditions the closure of the set 
{(Qi(x), . . . , Qt{x))\ x G Z rf — {0}} contains (0, ...,0). As a corollary, we 
deduce several results on the magnitude of the set A of g G GL(d,R) such 
that the closure of the set {(Qi(gx), . . . , Qt(gx)) : x G 7L d — {0}} contains 
(0, . . . , 0). Special cases are described when depending on the mutual position 
of the hypersurfaces {Qi = 0}, i = 1, . . . , t, the set A has full Haar measure or 
measure zero and Hausdorff dimension d 2 — 



1 Introduction 

It was conjectured by Oppenheim that if 

^ ClijXiX j , X (-^l; ■ ■ ■ ; 3Cd) 



is a real nondegenerate indefinite quadratic form in d variables, d > 5, which is not 
proportional to a rational form, then 

\/e>0 3xeZ d -{0}: \Q(x)\ < s. (1) 

This conjecture was proved by Margulis [Ma89j. In fact, he proved that if Q is 
a real nondegenerate indefinite quadratic form in d variables, d > 3, which is not 
proportional to a rational form, then 

> 3x E 7L d - {0}: < \Q(x)\ < e. 

Sec [Ma97 j for an up-to-date survey. It is also known that (0) fails for some indefinite 
irrational forms in 2 variables. For example, (0) fails for Q = x\ — o?x\ with a badly 

*This article is a part of author's PhD thesis at Ohio State University done under supervision of 
Prof. Bergelson. 
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approximable number a (e.g. a = 1 + a/2). Recall that a number a is called badly 
approximable if there exists c > such that for all integers p and q > 0, 

c 

k?« — P > -• 

In this paper we study similar questions for systems (Qf j = 1, . . . , t) of quadratic 
forms. Namely, we would like to know when the following statement is true: 

Me > 3x EZ d - {0}: max \Qj(x)\ < e. (2) 

j=i,...,t 

We expect that an analog of the Oppenheim conjecture holds for systems of quadratic 
forms (see Conjecture HH1 below). However, the proof of this conjecture seems to be 
beyond the reach of available methods. 

One can give a convenient characterization of property (j2J for systems quadratic 
forms in two variables: (J2J) holds for nondegenerate indefinite quadratic forms 

Qj(xi, x 2 ) = (djXi + bjX 2 )(cjXi + djX 2 ), j = 1, • • • , t, 

iff the set 




contains a number which is not badly approximable (we consider oo to be not badly 
approximable). In the case of one quadratic form, this was observed by Dani [DaOO . 
Theorem Q below is an analog of this fact in dimension d > 2. 

Validity of (0) depends on the common position of the hypersurfaces {Qj = 0}, 
j = l,...,t. For example, it is easy to see that if these hypersurfaces intersect 
only at the origin, then (j2j) fails. We investigate the case when the intersection of 
the hypersurfaces {Qj = 0}, j = l,...,t, is still relatively small: they intersect 
transversally in a strong sense. To formulate our result, we recall that a vector 
a = (q;i, . . . , ad) G M. d is called well approximable of order r (r > 0) if there exist 
p n G Z d and q n > such that 

q r n ■ \\q n a — p n \\ — > as n — > oo. (3) 

Theorem 1 Let Qj, j = l,...,t, be nondegenerate indefinite quadratic forms in 
d variables, d > 3. Suppose that the intersection of the hypersurfaces {Qj = 0}, 
j = 1, . . . , t, is not {0}, and at every point of intersection, which is different from 0, 
the space spanned by the normal vectors to the hypersurfaces has dimension d — 1. 
Then (0) holds iff the intersection of the hypersurfaces {Qj = 0}, j = l,...,t, 
contains a vector v G M. d such that for some j = 1, . . . ,t, Vj ^ 0, and the vector 

i 7^ j^j G is well approximable of order one. 
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We also study the case when zero hypersurfaces of the quadratic forms have a 
common tangent plane at a point of intersection. Choose and fix a line along which 
the hypersurfaces {Qj — 0}, j — 1, . . . , t, have a common tangent plane. Let fi be 
the direction vector of this line. We choose a basis % = 1, . . . , d} such that the 
vector f 2 is outside of the tangent plane, and fi, i = 3, . . . , d, are in the tangent 
plane. Define a t G SL(<i, R) by 

aji = e~*/i, aj 2 = e l f 2 , atfi = fi, i = 3, . . . , d. (4) 

The following theorem establishes a connection between properties of the semiorbit 
of a t , t > 0, in SL(d, R)/SL(d, Z) and Diophantine condition (J2J). First results of this 
type were discovered by Dani |Da85t lDa86j. 

Theorem 2 Lei Qj, j = 1, . . . ,t, be nondegenerate indefinite quadratic forms in d 
variables, d > 3. Suppose that the hypersurfaces {Qj = 0}, j = l,...,t, have a 
common tangent plane at a point of intersection, which is different from 0. If the 
semiorbit 

{a t SL(d,Z): t > 0} C SL(tZ,R)/SL(d,Z), 

where a t is defined in is not relatively compact, then holds. Moreover, if for 
some aj G R, j = 1, . . . ,t, the quadratic form Ylj a jQj ^ s definite of rank d — 1, 
then (0j implies that the semiorbit {a t SL(d, Z): t > 0} is not relatively compact in 
SL(d,R)/SL(d,Z). 

Remark Since the choice of the vectors /j, % = 2, . . . , d, is not unique, the transfor- 
mation a t is not uniquely defined. Nonetheless, one can check that the property that 
the positive semiorbit of a t is bounded is independent of a chosen basis. 

Theorems ^ and 121 allow us to investigate how often (0) holds in the space of 
systems of quadratic forms. For quadratic forms Qj, j = 1, . . . , t in d variables, we 
consider a parametric family 

(()",:. i 1 /!• geGL(d,R), 

where Q 9 j{x) d = Q(gx), and denote by A(Qi, ■ ■ ■ ,Qt) the set of g G GL(d, R) such 
that (QJ: j = 1, . . . , t) satisfies ©. 

Corollary 3 Let Qj, j = 1, . . . ,t, be nondegenerate indefinite quadratic forms in d 
variables, d > 3, and A = A(Q±, . . . , Qt). 

(i) If the intersection of the hypersurfaces {Qj = 0}, j = 1, . . . ,t, is not {0}, and 
at every point of intersection, which is different from 0, the space spanned by the 
normal vectors to the hypersurfaces has dimension d — 1, then A has measure 
zero and Hausdorff dimension d 2 — 
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(ii) If the hyper surfaces {Qj — 0}, j — 1, . . . , t, have a common tangent plane at a 
point of intersection, which is different from 0, then A has full Haar measure 
in GL(d, R) (i.e. its complement has measure zero). Moreover, if for some 
ctj gRj' = 1,..., t, the quadratic form a jQj is definite of rank d — 1, then 
the complement of A in GL(d, R) has Hausdorff dimension d 2 . 

Recall that the radical Rad(Q) of a quadratic form Q, which is defined on a 
vector space S, is the subspace of vectors in S that are orthogonal to S with respect 
to Q. 

We prove the following result that complements Corollary EJii) : 

Theorem 4 Let Qj, j = 1, . . . ,t, be nondegenerate indefinite quadratic forms in d 
variables, d > 3, and A = A(Qi, . . . , Q t ). Suppose that for some ^ 6 1, j = 2, . . . , d, 

V= f p| Rad^- - 

j=2,...,d 

has dimension at least 2, and V contains a vector v ^ such that Q\{v) = 0. 

(a) Unless dimV = 2 and the quadratic form Q\\v is nondegenerate, the comple- 
ment of A in GL(d, M.) is contained in a countable union of proper submanifolds 
of dimension at most d 2 — d + 1. 

(b) If dim V = 2, the quadratic form Q\\v is nondegenerate, and for some ctj G R, 
j = l,...,t, the quadratic form 'Yl l j a jQj ^ s definite of rank d — 2, then the 
complement of A in GL(d,R) has Hausdorff dimension d 2 . 

Note that the conditions of Theorem 0] imply that the hypersurfaces {Qj = 0}, 
j = 1, . . . , t, have a common tangent plane at v. In particular, it follows from 
Corollary Efii) that the complement of A has measure 0. 

The simplest example for Theorem Ufa) is provided by the system of quadratic 
forms 

Qj (a?) = + + cijX^, j = 1, . . . , t. 

It follows from the result of Dani and Margulis DM90, that the complement of 
A(Qi, Q 2 ) is a countable union of proper submanifolds of GL(3, M). 

It is easy to refine Corollary El and Theorem 0] as follows. Denote by A' the set of 
g G GL(d,M) such that 

ye > 3x G Z d - {0}: < max \Qj(gx)\ < e. (5) 

j=i,...,t 

Corollary 5 Statements of Corollary^ and Theorem^are valid for A'. 

The next section is devoted to the proofs of stated results. Some open problems 
are discussed in Section 3. 

Acknowledgments: Thanks are due to my advisor V. Bergelson for many 
stimulating and enlightening discussions. I am also grateful to the referee for several 
pertinent suggestions, which have significantly improved the quality of this paper. 
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2 Proofs 



Proofs of Theorems ^ and |2 involve only elementary linear algebra. Theorem ^follows 
from Lemmas [7| and |H1 below. To prove Theorem we reduce it to a question about 
a system of linear forms. The proof of Corollary Ef i) uses Lemmas U\ and El and 
the result of Jarmk |,Ia31| . Corollary Efii) and Theorem 0] are deduced from known 
properties of certain flows on the space of unimodular lattices. 

If it is not stated otherwise, || • || denotes the Euclidean norm on M. d . 

The following three lemmas are used in the proof of Theorem ^ 

Lemma 6 Let Qj, j = l,...,t, be nondegenerate indefinite quadratic forms in d 
variables, d > 3, whose zero hyper surfaces intersect along a line £, and the span 
of normal vectors to the hyper surf aces {Qj = 0}, j = 1, . . . ,t, at a point of I has 
dimension d — 1. Denote by tt the orthogonal projection on the line i. For m, M, 
e > 0, put 

R m ,M = {x eR d : \\n(x)\\ > M,\\x - n(x)\\ <m\\n(x)\\}, 

T(e) = {x E R m ,M'- H 71 "^)!! • 11^ — 7r(^)|| < £ }i 

S(e) = {x e Rm, M - \Qj{x)\ < e,j = 1, . . .,*}. 

Then there exist m, M, c\, c-i > such that for every e e (0, 1), 

T( Cl e) C S(e) C T(c 2 e). 

Proof It is enough to prove the lemma for a system of quadratic forms Qj, j = 
1, . . . , d — 1, such that normal vectors to {Qj = 0}, j = 1, . . . , d — 1, at a point of £ 
are linearly independent. 

For a vector x = (x\, . . . , Xd) G M. d , put x = {x\, . . . , x^-i)- 

Using a change of variables, we can transform I to the line {xi — 0: i — 1, . . . ,d — 
1}. Thus, it is enough to consider the case when the line I is as above. Then 

Qi(x) = qi(x) +x d Li(x), i = l,...,d-l. (6) 

Note that Lj is independent of Xd because Qi(0, . . . , 0, 1) = 0. The tangent plane to 
{Qi = 0} at (0,...,0,1) is {L l = 0}. 
Let 

T(a, e) = T{e) n {xd = a} and S(a, e) = S(e) R {xd = a}. 
For \a\ > M, we have 

T(a,e) = {(x,a): \a\ ■ \\x\\ < e, \\x\\ < m\a\}, (7) 

S(a, e) = {(x, a): \qi{x) + aLi(x)\ < e, i — 1, . . . , d — 1; \\x\\ < m\a\} . (8) 
First, we claim that for fixed sufficiently small m > 0, 

diam(S(a,£)) -> (9) 
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uniformly one 6 (0, 1) as \a\ — ► oo. Let L = (Z^) be a linear map where kj are the 
coefficients of the linear forms Lj. Note that L is nondegenerate because the normal 
vectors to {Qj = 0}, j — 1, . . . , d — 1, at a point of £ are linearly independent. Using 
© and (jHJ), we have 

|aL;(x)| < e + \qi(x)\ < 1 + l3i\\x\\ 2 < 1 + Ara|a| ■ (10) 

for x G S(a, e) and i = 1, . . . , d — 1. 

Suppose that there exists a sequence (u^, a n ) G S(a n ,e) with \a n \ > 1, and 
||w^|| — > oo. We may assume that — >■ t> for some t> on the unit sphere S 1 ^ 2 . 
From (JTUJl we get 

|^(w (n) )| < + Am||u (n) || < 1 + Am||u (n) ||, i = l,...,d-l. 

| On | 

It follows that 

\Li(v Q )\ < Piin, i = l,...,d-l. (11) 

Take 

m = -min{max{/5r 1 |L i (a;)|: z = 1, . . . , d - 1} : x G S d ~ 2 } . (12) 

Note that m > because Lj, z = l,...,d— 1, are linearly independent. Then 
the equations (Jllj) and (|12|) contradict each other. This shows that diam(S(a, e)) 
is bounded for s G (0,1) and a > 1. From the first part of (fTUj). we see that 
|a| • ||£(x)|| for x G S(a,e) is bounded too. Since L is nondegenerate, for some c > 0, 
diamfSYa, e)) < -A when e G (0, 1) and a > 1. This proves the claim @. 

|a| 

Consider a transformation it: R ' -1 — > defined by 

""i = qi(x) + aLi(x), i — 1, . . . , d — 1. 

By©, 

S(a,e) = {(x, a): \ui(x)\ < e,i = 1, . . . , d — 1; ||x|| < m|a|} . (13) 

The Jacobian of the transformation w is J = (j^pj + ctL. Note that det J(0) = 

det L ^ 0. Therefore by w is a diffeomorphism on S"(a, e) for sufficiently large \a\. 
Let g = dx\ + ■ ■ ■ + dx 2 d _ 1 be the standard Riemann metric, u*g = du\ + - ■ ■ + du 2 i _ 1 be 
the pull-back of g under the transformation u, and L*g be the pull-back of g under 
L. Clearly, for some r\,ri > 0, 

rig < L*g < r 2 g. 

Since a -1 J — > L as \a\ — > oo uniformly on compact sets of ^-§- — > L*g uniformly 

on compact sets. Therefore, 

, u*g . v 

< —5- < ^ (14) 
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on S(a,e) for \a\ sufficiently large (say \a\ > M). Note that ||x|| is the distance to 
the origin with respect to the metric g, and ||tt(a?)|| is the distance to the origin with 
respect to u*g. Therefore, it follows from ()14J) that for x G S(a,e) and \a\ > M, 

1/2 ^ || / N,| , 1/2 

r i \o\ ■ ||x|| < ||w(xj|| < r 2 |a| ■ ||x||. 
By (0) and (JBJ), for |a| > M, 

T [a,r 2 1,2 e^j C 5(0, e) C T fa, r^ 1/2 v^e) . 

The lemma is proved. □ 

Lemma 7 For a line i in M. d , denote by 7i£ the orthogonal projection on the line 
i. Assume that zero hyper surfaces of a system of nondegenerate indefinite quadratic 
forms Qj , j = 1, . . . ,t, in d variables intersect at a point, which is different from 
0, and at every nonzero point of intersection, the span of the normal vectors to 
these hypersurfaces has dimension d — 1. Then holds iff there exist a line £ 
in the intersection of the hypersurfaces of {Qj = 0}, j = 1, . . . ,t, and a sequence 
x (n) EZ d - {0} such that 

\\Mx (n) )\\ -> oo and \\Mx {n) )\\ • P (n) - M^ n) )\\ -> 0. (15) 

Proof Let m, M, ci, C2 > be as in Lemma El 

Suppose that Qj, j = 1, . . . , t, satisfy (J2J). Then there exists a sequence x^ G 
Z d - {0} such that 

e n = f max{|Q,(xW)|: j = 1, . . . , t} - 0. (16) 
Passing if needed to a subsequence, we may assume that 

S W =' pgji - ft 

for some y on the unit sphere. Then Qj(yo) — 0, j = 1, . . . , t. Let £ be the line 
through f/o and the origin. 

If \\x^ || ^> oo, we can take {x^} to be bounded. Then for each j, Qj(x^ n ') admits 
only finitely many values for n > 1. Therefore, by (|lf))l. Qj(x^ n ') = 0, j = 1, . . . , i, 
for sufficiently large n. We can replace x^ by nx^ n \ Thus, we may assume that 
||x( n )|| — > oo. 

Since f/( n ) — ► f/ , the sequence x^™^ is inside of the cone 

\\X - 7^(x)|| < m||7Tf(x)|| 

for sufficiently large n. It follows that if ||7r^(x ( - n ^) || -■*> oo, then ||x*- n ^|| oo. There- 
fore, 7T£(x ( - n - ) ) — > oo. In particular, x^ 1 ' G R m ,M for sufficiently large n. By Lemma 

El 

\\7r t {x^)\\-\\x^-TT £ {x^)\\<c 2 e n ^0. 
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Conversely, suppose that (fTK|) is satisfied for some line i in the intersection of the 
hypersurfaces {Qj = 0}, j = l,...,t. Then for sufficiently large n, x^ 1 ' G R m ,M- 
Applying Lemma El we get 

\Qi& n) )\ < c^WMx^W • - n& n) )\\ - o, j = i, . . . , t. 

This proves the lemma. □ 

Lemma 8 Let v — (v\, . . . , Vd) G M. d with Vj ^ for some j = 1, . . . , d, and tt is the 
orthogonal projection on the direction v. Then the vector i ^ jj G W 1 ^ 1 is well 
approximable of order 1 iff for some sequence x^ n ' G Z, d — {0}, 

||7r(x (n) )|| -> oo and \\ir{x {n) ) || • ||x (n) - vr(x (n) ) || -> 0. (17) 

Proof Without loss of generality, j = 1. Put aj = z = 1, . . . , d — 1. 

Let Lj(x) = — x i+ i, i = 1, . . . , d — 1. Note that the planes defined by = 0, 
z = 1, . . . , d — 1, intersect along the line in direction v. 

Suppose that («i, . . . , o^-i) is well approximable of order 1. There exists a se- 
quence x^ 1 ' G Z d such that 



»i 



\Li (x {n) )\ ^ 0, i = l,...,d-l, 



with 7^ for all n. It follows that 

Li(x {n) ) ->0, z = l,...,d-l. (19) 

Suppose that is bounded. A linear map A: M d -> R d , which is defined by 

(Ax)i = Li(x), % — 1, . . . , d — 1, and (Ax)^ = xi, (20) 

is nondegenerate. Hence, by (fTHj) . x^ is bounded, and Li(x n ) = 0, z = 1, . . . , cf — 1, 
for sufficiently large n. In this case, (117)1 holds for the sequence nx'™'. 

If |xj | is not bounded, we may assume that — > oo. By ()19|). for sufficiently 
large n., 

/d-l \ V2 

\\Ax (n) \\ = I J2Li{x (n) ) 2 + (4 n) ) 2 j < Vd\x^\. (21) 

Since A is nondegenerate, for some c\ > and every x G M d , 

lk(x)|| < Cl ||Ax||. (22) 

Let P be the plane through the origin orthogonal to v. A map C: P — > M d_1 defined 
by 

(Cx)i = Lj(x), z = 1, 
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is invert ible. Therefore, for some c 2 > and every x G P, 

|| x — 7r(x)|| < c 2 max{|Lj(x)|: i = 1, . . . ,d — 1}. (23) 

In fact, the last inequality holds for every x G M. d because it is independent of 
translations by vectors parallel to v. Now jnj) follows from (JIBJ), (j2U), (E2), (EH) - 

Conversely, suppose that (fT7j) holds for some sequence a;*™) G Z d — {0}. Note that 
for x G M d , |-£/i(x)| is up to a constant the distance from x to the plane {Li = 0}, and 
||x — tt{x)\\ is the distance from x to the line through v. Therefore, for some > 
and every a; G W 1 , 

\Li{x)\ < ri \\x - 7t{x)\\, t = l,...,d-l. (24) 

In particular, it follows that 

L t (x (n) ) -> 0, i = 1. (25) 

Clearly, 7^ for sufficiently large n. 
Consider a linear map i? defined by 

(Bx)i = Li(x), i — 1, . . . , d — 1, and (Bx)d = ir(x). 

Since -B is nondegenerate, for some r% > and every x G M d , 

|xi| < r 3 ||i?x||. (26) 

By (|T7j) and (|23j) . for sufficiently large n, 

/d-l \ V2 

115x^11 = K^L 4 (x (n) ) 2 + ||7r(x (n) )|| 2 < \/d||7r(xW)||. (27) 

Finally, (JTEJ) follows from (j23), and (J2ZJ) - Thus, the vector (e*i, . . . , a d -i) 
is well approximable of order 1. □ 

Combining Lemmas [3 and IHl we deduce Theorem ^ 

The proof of Corollary |Sfi) uses Lemma [3 Lemma |Hl and the result of Jarnik on 
Hausdorff dimension of the set of well approximable vectors |Ja31j . It is known that 
the intersection of the set of well approximable vectors of order r in R d with every 
nonempty open set has Hausdorff dimension 

Proof of Corollary Let Q be the projective variety in the complex projective 
space P d_1 (C) defined by Qj = 0, j = 1, . . . , t. Since at every point of Q(M) the rank 
of Jacobian g^''"'^ is equal to d — 1, it follows that every irreducible component of 
Q that has nonempty intersection with Q(R) has dimension zero. Therefore, Q(K) 
consists of finitely many points. This means that the zero hypersurfaces {Qj = 0}, 
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j = l,...,t, in l d intersect along only finitely many lines that pass through the 
origin. Let v s , s = 1, . . . , N, be the direction vectors of these lines. Then zero 
hypersurfaces of Q|, j = 1, . . . , t, intersect along vectors s = 1, . . . , N. For 

some g s G GL(d, R), v s — g s e\, where e\ = (1, 0, . . . , 0). We have 

g~^ s = {{.g~ l g s )iu ■ (g~ 1 g s )di) , s = i,...,N. 

By Lemma 

N 

A=\jA s , 

s=l 

where A s is the set of g G GL(d, M.) such that ()15|) holds for the line £ through the 
origin in direction g~ 1 v s . Clearly, it is enough to prove the theorem for each of the 
sets A s separately. 

Fix s = 1, . . . , N. Let 

G a ,i = {geGL(d,R):(g- 1 g a ) il ^0}, 

and A Sj i = A s n G s j for i — 1, . . . , d. 

By Lemma |Hl for g G G s i, g belongs to A si iff the vector 



(9 1 9s)ji 



J i 



s il 



is well approximable of order 1. Consider maps F s ^. G s ^ — > M(d,M), i = 1, . . . ,d 
defined by 



\g gsjii 

F sAg)hh = {g^gs)^ for Uuh) i {{j, l), j + %)■ 

Note that F s ^ is a diffeomorphism on G s ^, and F a ^(G s ^) is a complement of a finite 
union of hypersurfaces in M(d, R). Let Wi be the set of g G M(cf,R) such that the 
vector (gji: j ^ i) is well-approximable of order 1. We have that for g G G s>i , g 
belongs to A s>i iff F S)i (g) belongs to Wi, i.e. A s>i = F~l (Wi fl F S:i (G S)i )). Therefore, 

d d 

A s = |J A S}i = |J F- 1 (W t n F., 4 (G.,,)) • (28) 

i=l j=l 

The intersection of the set of well approximable vectors of order 1 in M.^ 1 with 
every nonempty open subset has Hausdorff dimension |. It follows that the inter- 
section of the set Wi, i — 1, d, with every nonempty open subset of M(d, R) has 
Hausdorff dimension d(d — 1) + 1 + ^ = d 2 — (see |BD| Section 3.5.5]). Since 
Fs,i(G s ,i) is open in M(d, R), the set Wi fl F Sji (G S:i ) has the same property. There- 
fore, by (J2HJ), the intersection of A s with every nonempty open subset of M(d, R) has 
Hausdorff dimension d 2 — ^p. □ 
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The following two lemmas are crucial for the proofs of Theorem El and Corollary 
IHfii). The proofs of the lemmas are based on ideas of Dani |Da85| IDa86 . 

Lemma 9 Let Li, i — 1, ... ,d, be linearly independent linear forms in d variables, 
d>2. Define a t G SL(d,R) by 

Lx{a t x) = e~%(x), (29) 
L 2 (a t x) = e t L 2 (x), (30) 
Li(a t x) = Li(x), i = 3,...,d. 

Then the set {a t SL(d, Z): t > 0} is not bounded in SL(d, R)/SL(d, Z) iff there exists 
a sequence x^ 1 ' G Z d — {0} such that 

L 1 (x ( " ) )L 2 (x (n) ) -> and Li(x (n) ) -> 0, i = 2, . . . , d. (31) 

Proof Let || • || be the norm on R d defined by 

||x|| = max{|Lj(x)|: i = 1, . . . , d}, x G R d . 

By Mahler Compactness Criterion, the orbit {a t SL(d, Z): t > 0}, is unbounded iff for 
some t n > and x^ G Z d - {0}, a tn x M -> 0, i.e. 

=max{e-* B |L 1 (x( n ))|,e*"|L 2 (xW)|, (32) 
|L«(3z (n) )|:i = 3,...,d} ->0. 

It is clear that (jS2)) implies (f3~T|) . Conversely, suppose that (JHU holds. 

First, we consider the case when L 2 (:£^) = for infinitely many n. Taking a 
subsequence, we may assume that L 2 (x (n) ) = for all n. Then ® holds for every 
sequence t„ — > oo such that 

e-*"|Li(xW)| -> 0. 

Now we may assume that L 2 (x(™)) 7^ 0. If Li(x ( - n ^) is bounded, then since the 
linear forms Li, i = 1, . . . , d, are linearly independent, x^ 1 ' is bounded. This implies 
that for sufficiently large n, L 2 {x^ n ') = 0. Thus, we may assume that \L\ 
Then (J321) holds for the sequence t n such that e tn = |Li(x (n) )| 1/2 • |L 2 (x (n) )|- 1/2 . Note 
that t n > for sufficiently large n. This proves the lemma. □ 

Lemma 10 Let Li, i = 1, . . . , d, be linearly independent linear forms in d variables, 
d>2. Then the set $ of g G GL(d, R) such that 

L 1 (gx (n) )L 2 (gx {n) ) -> and Li{gx {n) ) -> 0, i = 2, . . . , d. (33) 

/or some sequence G Z d — {0} /ias /uZZ i/aar measure, and its complement has 
Hausdorff dimension d 2 . 
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Proof Let L^(x) — Xi, i — 1, . . . , d. For some go G GL(d, R), Li(gx) = L®(g gx), 
i = l,...,d. Since multiplication by g is a diffeomorphism of GL(d, R), we may 
assume without loss of generality that g = 1 and Li — L®, i — 1, . . . , d. 
For every matrix u G GL(d, R), 

u = diag(A, 1, . . . , 1) • v 

with A G R x and v G SL(d, R). Moreover, the map 

GL(d,R) -> R x x SL(d,R): u h+ (A,v) 

is a diffeomorphism. Since 

L\{ux) = XLi(vx), Li{ux) = Li(vx), i — 2,...,d, 

(}3*3*j) holds for g — u iff it holds for g = v. Therefore, if we show that the set 
$ fl SL(d, R) has full measure in SL(d, R), this will imply that the set $ has full 
measure in GL(d,R) too. Also by the argument as in \BD\ Section 3.5.5], if the set 
SL(d, R) - ($ n SL(d, R)) has Hausdorff dimension d 2 - 1, then the set GL(d, R) — $ 
has Hausdorff dimension d 2 . Hence, the proof reduces to a question about SL(d, R). 
By Lemma El ()33|) holds for some sequence x^ G Z d — {0} iff the set 

{^a^SL^Z): t > 0} 

is unbounded in SL(<i, R)/SL(<i, Z). This is equivalent to the set {a t gSL(d, Z): t > 0} 
being unbounded. It is known that the set of g G SL(g?, R) such that the orbit 
{a t gSL(d, Z): t > 0} is bounded has Haar measure zero (Moore Ergodicity Theo- 
rem [ZI1 Theorem 2.2.6]), and its intersection with every nonempty open subset has 
Hausdorff dimension d 2 — 1 (Kleinbock, Margulis KM96J). Therefore, the set of 



g G SL(d, R) such that (J3l?j) holds for some sequence x^ G Z d — {0} has full measure, 
and its complement in SL(d, R) has Hausdorff dimension d 2 — 1. This implies the 
lemma. □ 



Proof of Theorem [2] With respect to the basis {ff. i = 1, . . . , d}, 

Qj(y) = qj(y2,---,yd) + yiij(y2,---,yd), j = i,...,t, (34) 

where qj is a quadratic form, and lj is a linear form. Note that lj is independent of y\ 
because Qj(fi) = 0. The tangent plane to {Qj = 0} at f\ is {lj = 0}. Thus, lj = Cjl, 
j = 1, . . . , t, for some Cj G R — {0} and a linear form /. Since f\ and fi, i = 3, . . . ,d, 
are in the tangent plane, l(y) = y<i- Let Li(x), i = 1, . . . , d, denote the coordinates 
of vector x with respect to the basis {ff % = 1, . . . , d}. Then 

Qj{x) = qj(L 2 (x), . . .,L d (x)) + c j L 1 (x)L 2 {x), j = 1, . . . ,t, 
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and holds. It follows that ((2)) holds for {Qy j = 1, . . . , t) provided that for some 
sequence x^ G Z d — {0}, 

L!(x (n) )L 2 (x (n) ) -> and L;(x (n) ) -> 0, z = 2, . . . , d. (35) 

Thus, the first statement of the theorem follows from Lemma El 

Suppose that Q = J2j a jQj * s definite of rank d — 1. Since Q is definite, it follows 
from (|3*3)l that = 0(2/2, • • • , Vd)- Therefore, (J2J) implies (j3*5Jl for some sequence 
x*™) £ Zi d — {0}, and the second statement of the theorem follows from LemmaOH □ 

Proof of Corollary El(n) It was shown in the proof of Theorem El that (J2J) holds 
for (Qj : j = 1, . . . , t) provided that for some sequence G Z d — {0}, 

Li(</5 (n) )L 2 (</» (n) ) -»• and L,(^ (n) ) -> 0, z = 2, . . . , d, (36) 

and when a linear combination of Qj, j = 1, . . . , d, is definite of rank d — 1, j = 
1, . . . ,t) satisfies (J2J) iff holds. Thus, Corollary |Hfii) follows from Lemma |0l □ 

To prove Theorem Ufa), we use the following lemma: 

Lemma 11 Lei Qj, j = 1, . . . , t, be a system of quadratic forms that satisfies the 
conditions of Theorem^a). Then there exists a nontrivial one-parameter unipotent 
subgroup ofSL(d, R) which leaves all Qj, j = 1, . . . ,t, invariant. 

Proof Let Qi\y be nondegenerate. Then diml 7 > 3. Write 

R d = V © W 

where W is the orthogonal complement of V with respect to Q\. Then the group 

H = / SO(gi|y)©zd l y 

leaves Q\ invariant. Since V is in the radical of Qj — (3jQi, the forms Qj, j — 2, . . . , t 
are invariant under H too. The group if is a noncompact semisimple Lie group. 
Hence, it contains a nontrivial one-parameter unipotent subgroup. This proves the 
lemma when Qi\y is nondegenerate. 

Now suppose that Qi\y is degenerate, and Qi\y ^ 0. Choose a basis {ff % = 
l,...,d} of R d such that fi is in the radical of Qi\y, f% is in V, Qi{f-2) ^ 0, and 
fi, i = 3, . . .,t, are orthogonal to / 2 with respect to Q 1 . Since / 2 is in the radical 
of Qj — PjQi, j = 2, . . . , t, J2 is orthogonal to /», i = 3, . . . , t, with respect to Qj, 
j = 2, . . . , t. In the basis {/»: i = 1, . . . , d}, 

Qj(y) = PAvl + yiHvs, Vd)) + qjivs, ■■■,yd), j = i,...,t 
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for some linear form L and quadratic forms qj, j — 1, . . . , t. (Here we put f3\ = 1.) 
Note that L is the same all j because f\ is in the radical of Qj — (3jQ\, j — 2, . . . , t. 
Define a linear transformation Ut G SL(d, M) by 

u t : Vx^yi- 2ty 2 - t 2 L(y 3 , ...,y d ), 

ut ■ y 2 ^y 2 + tL(y 3 ,...,y d ), 

u t : yi h-> y u i > 3. 

Then {uf. t G R} is a nontrivial one-parameter unipotent group that stabilizes Qj, 
j = l,...,t. 

It remains to consider the case when Qi|v — 0. Choose a basis {/j: i = 1, . . . , d}, 
such that fx and / 2 is in V. With respect to this basis, 

Qj(y) = Pj(yi L i(y3, ■■■,y c d + 2/2-^2(2/3, •••,%)) + ^(2/3, •••,%) 

for some linear forms L\ y L 2 , and quadratic forms qj, j = l,...,t. (Here we put 
Pi = 1.) Note that L x and L 2 are independent of j because fi and f 2 are in the 
radical of Qj — (3jQ\, j = 2, . . . , t. The linear forms L\ and L 2 are not zero because 
the quadratic form Q\ is nondegenerate. Define a linear transformation vt G SL(d, K) 
by 

^ : y 1 ^y 1 +tL 2 (y 3 ,...,y d ), 
v t ■■ y 2 ^y 2 -tL 1 (y 3 ,...,y d ), 
v t ■ yi >-> « > 3. 

Then t G M} is a nontrivial one-parameter unipotent group that stabilizes Qj, 
j = 1, . . . , t. We have proved the lemma. □ 

The following Lemma is used in the proof of Theorem EJ^b). Its proof is essentially 
the same as the proof of Lemma 01 and is omitted. 

Lemma 12 Let Li, i = 1, . . . , d, be linearly independent linear forms in d variables, 
d>2, and a t is defined as in \29^l . Then the set {a t SL(d, Z): t G K} is not bounded 
in SL(d, R)/SL(d, Z) iff there exists a sequence x^ n > G Z d — {0} such that 

L l (x {n) )L 2 (x {n) ) -> and L t (x in) ) -> 0, i = 3, . . . , d. (37) 

Proof of Theorem H Let G = SL(d, E) and T = SL(d, Z). 

To prove (a), we show first that the complement of A fl G in G is contained in 
a countable union of submanifolds of dimension at most d 2 — d. Denote by U C G 
a one-parameter unipotent subgroup that stabilizes all Qj, j = l,...,t. Such a 
subgroup exists by Lemma [TTJ By Ratner's topological rigidity [Ra9lj, the set Q of 
g G G such that UgT is not dense in G/Y is a countable union of sets of the form FY 
where F is a connected Lie subgroup of G. It is known that the maximal dimension 
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of a proper connected subgroup of G is d 2 — d |()V| Sec. 3.3]. By Mahler compactness 
criterion, for every g e G — Q, there exist u n G U and & n ' G Z d — {0} such that 
u n gx^ — > as n — > oo. Then 

Q^W) = QiiungxW) _ o, j — 1, . . . ,t. 

Therefore, G — A fl G is contained in fl. 

Let GL ± (ci,R) be the group of matrices with positive/negative discriminant. It 
is clear that 

A n GL + (d, R) = [J A • (A n G) (38) 

A>0 

Therefore, the complement of AnGL + (<i, R) in GL + (d, R) is contained in a countable 
union of submanifolds of dimension at most d 2 — d + 1. Similarly, 

A n GL~(d, R) = |J Xg • (A n G) (39) 

A>0 

for a fixed go G GL _ (<i, R) and A d = A(Qf > , . . . , Qf°). It follows that the complement 
of A R GL~(d, R) in GL~(d, R) is contained in a countable union of submanifolds of 
dimension at most d 2 — d + 1. This proves (a). 

Now we prove (b). Take a basis {ff i = 1, . . . ,d} such that V = (/i, /2), and 
/j, z = 3, . . . , d is orthogonal to U with respect to Q\. It follows from the definition 
of V that fi, i = 3, . . . ,d are orthogonal to V with respect to Qj, j = 2, . . . , t, too. 
In addition, we can change f\ and fi such that with respect to the basis {ff i = 
l,...,d}, 

Qj(V) = P3V1V2 + qj(ya, ■■-,yd), 3 = h'~,t (40) 

def ^ 

for some quadratic forms qj, j = 1, . . . , t. (Here we put /3i = 1.) Let Q = J2j a jQj 
be a definite form of rank d — 2. It follows from (pfUj) that Q(y) = Q(y3, ■ ■ ■ ,Vd)- 
Denote by Li(x), i = 1, . . . , d, the coordinates of a vector x with respect to the basis 
{ff i = 1, • • • , d}. Then (J2J) holds for (Q 9 -: j = 1, . . . , t) with g 6 G iff for some 
sequence x^ n ^ G Z d — {0}, 

L x (^ (n) )L 2 (^ (n) ) and U{gx {n) ) -> 0, i = 3, . . . , d. 

By Lemma ^1 this is equivalent to the set {g^atgT: t G R} being unbounded in 
G/r. Therefore, by the result of Kleinbock and Margulis [KM96 , the set G — (AnG) 
has Hausdorff dimension d 2 — 1. 

To show that the complement of A in GL(d, R) has Hausdorff dimension d 2 , one 
may use (JHHJ), (JHHJ), and [BDj Section 3.5.5]. This proves (b). □ 

Proof of Corollary 0>] Under the conditions in (i), the intersection of hypersurfaces 
Qj — 0, j = 1, . . . , t, consists of finitely many lines that pass through the origin. 
Denote by v s , s = 1,...,N, the direction vectors of these lines. Note that for 
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g G GL(d, R), hypersurfaces Q| = 0, j = 1, . . . , t, intersect along lines in directions 
g~ l v s , s = 1, . . . , iV. For s — 1, . . . , iV and x G R d — {0}, denote by R s (x) the set of 
g G GL(d, R) such that g v s is collinear with x. The set R s (x) is a submanifold of 
dimension <i 2 — (d — 1). Since 

A-A'C |J R s {x), 

l<s<N; zeZ d -{0} 

the set A — A' has Hausdorff dimension at most d 2 — (d — 1) < d 2 — This shows 
that the set A' has Hausdorff dimension d 2 — < ^ L . 
To prove (ii), we define 

Kj(x) = {g G M(d, R): Q^z) = 0}, j = l,...,i,x6 R d . 

This is an algebraic subset of M(d, R), and for x^O, Kj(x) ^ M(d, R), which implies 
that it has measure 0. Note that 

A-A'C |J Kjix), (41) 
!<?<*; sez d -{o} 

where the last set has measure and Hausdorff dimension < d 2 . Therefore, when 
A has complement in GL(d,R) of measure zero, so does A'. The statement about 
Hausdorff dimension follows from (J4T| too. This proves (ii). 

We use notations from the proof of Theorem |3J If we show that G — A' n G is 
contained in Q, then it is possible to finish the proof as in Theorem^a). There exists 
x° G R d such that Qj(x°) ^ for every j — 1, . . . , t. Therefore, for every e > 0, there 
exists x £ G M. d such that 

< \Qj(x £ )\ <e, j = 1, . . . ,t. 

Write x e = g e e\ for some g e G G and e\ = (1, 0, . . . , 0). For g ^ Q, UgT = G. Thus, 
there exist u £ U and 7 G T such that 

< \Qj(usryei)\ = \Qj(g-yei)\ < e, j = 1, • ■ ■ ,t. 

Hence, g G A'. This show that G - A' H G C Q and proves Theorem Ufa). 

In the case (b), the intersection of the hypersurfaces {Qj = 0}, j = l,...,t, 
consists of two lines. Thus, by the same argument as in the proof of Corollary |3^i), 
the set A — A' has Hausdorff dimension at most d 2 — (d — 1). This implies part (b) 
of Theorem 0J □ 

3 Open Problems 

Theorems Q and |2] provide information about property (J2J only when the dimension 
of the space spanned by normal vectors to the hypersurfaces {Qj = 0}, j — 1, . . . , t, 
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is either 1 or d — 1. Clearly, this leaves a gap for d > 3. It is of interest to investi- 
gate property (J2J) assuming other conditions on the intersection of the hypersurfaces 
{Qj = 0}, j — 1, . . . ,t. At least, one should try to give a complete answer to the 
question about the magnitude of the set A(Qi, • • • , Qt), which was partially studied 
in Corollary El 

Corollary |3] illustrates that in some cases, property (j2J) has complicated Dio- 
phantine nature. This is analogous to the situation with Oppenheim conjecture in 
dimension 2. In higher dimensions, the following conjecture seems plausible: 

Conjecture 13 Let d G N be sufficiently large. Let {Q\,Q-i) be a pair of real nonde- 
generate quadratic forms in d variables such that every linear combination aQ\ + PQ2 
with a 2 +/3 2 7^ is indefinite, has rank > 3, and does not have all rational coefficients. 
Then holds. 

By a theorem of P. Finsler, for d > 3 every linear combinations aQ± + (3Q2 with 
a 2 + (3 2 7^ is indefinite iff the intersection of the hypersurfaces {Q\ = 0} and 
{Q2 = 0} is not equal to {0}. (This theorem was proved independently by many 
authors (see |Uh79j).) Therefore, under the conditions of Conjecture H21 the set 

{x: \Qi(x)\ < e,i = 1,2} 

is not compact. 

A question analogous to this conjecture was studied by Dani and Margulis [DM90 . 
They showed that if Q is a nondegenerate indefinite quadratic form, and L is a linear 
form in 3 variables such that the plane {L = 0} is tangent to the hypersurface 
{Q = 0}, and every linear combination aQ + f3L 2 with a 2 + (3 2 ^ does not have all 
rational coefficients, then {(Q(x), L(x)): x G M. d } is dense in R 2 . Similar result holds 
for a pair consisting of a linear form and a quadratic form in d variables, d > 4 |Coj . 

Some partial results towards Conjecture [THl were obtained in |Co73j by R. J. Cook. 
He studied pairs of diagonal quadratic forms in d variables, d > 9, with algebraic 
coefficients. 

Let us illustrate the conjecture by several examples: 

1. Let ~R d = Vi © V2 be a direct sum of vector spaces V\ and V2 of dimension 
at least 3. Let Qi and Q2 be indefinite quadratic forms such that Qi\vi is 
nondegenerate, Qi\vj = for % 7^ j, and V\ _L V2 with respect to Q\ and (^2- 
According to Conjecture HHJ © should hold for {Q\,Qi) provided that every 
nonzero linear combination of Qi and Q2 does not have all rational coefficients. 
This seems to be one of the easiest special cases of the conjecture. It should be 
possible to attack this case using the original approach of Margulis [Ma89] and 
Ratner's topological rigidity |Ra91j . 

2. Another promising case of Conjecture ITB1 is when a pair (Q\,Q2) of quadratic 
forms satisfies the conditions of Theorem HJa). By Lemma ITT1 both Qi and 
Q2 are invariant under a nontrivial unipotent subgroup. Hence, one can use 
Ratner's topological rigidity. 
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3. In general, the group of linear transformations that leaves both forms invariant 
may be finite. If this is the case, the method of Margulis, which is based on 
dynamics on homogeneous spaces of Lie groups, does not work. This problem 
can appear in every dimension even when the hypersurfaces {Q\ = 0} and 
{Q2 — 0} have a common tangent plane. For example, when 



up to a linear change of variables, the group that stabilizes both Q 1 and Q 2 is 
finite. 
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